INFO256 Project Report Implementation and Evaluation of Xtract in WordSeer

Natural languages are full of word collocations that frequently co-occur and correspond to arbitrary word usages. They appear in both technical and non-technical textual corpora and often have specific significance in individual contexts. Accurately retrieving and identifying collocations from a given corpus in an unsupervised manner is imperative to understanding and automatically generating text related to that corpus. Identifying collocations, however, is not an easy problem. They vary widely in length, in the words that are involved, and in the relations between the involved words. Moreover, words in a collocation can be adjacent or they can be separated by unrelated words. One straightforward way to retrieve collocations is to find the most frequent N -grams in a corpora. This approach is attractive because many collocations are just adjacent words that frequently appear together. WordSeer [1], a text analysis environment for literary corpora from UC Berkeley, uses a variation of theN -gram approach to identify collocations. Consequently, it faces the same shortcomings as anyN -gram-based approach would, namely, it cannot identify collocations with non-adjacent words, it has very high recall (and low precision), and it does not provide any functional information about the identified collocations. Xtract [2] is a statistical tool (i.e., a collection of algorithms) developed for identifying collocations with high precision and for statistically justifying their significance. It extends the N -gram approach with some statistical filters for better effectiveness. In this project, we have successfully implemented theXtract toolkit inWordSeer and compared its performance with the default N -gram-based collocation retrieval mechanism. Furthermore, we have evaluated and compared the performance of both approaches on two large text corpora: Shakespeare and Abstracts. The former is a literary collection of writings by William Shakespeare, and the latter is a technical corpora with abstracts collected from top HCI conferences. The key findings are the following: 1. Xtract is more precise than the N -gram approach. 2. In a show-of-hands poll over a group of 30 experts, Xtract outputs were preferred three out of four times. However, in all cases, a sizable fraction of the participants were undecided.